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SUMMARY 


Objective 

The  objectives  were  (a)  to  document  the  utility  and  utilization  of  advanced  instructional  features  (AIFs)  in  aircrew 
training  devices  (ATDs),  (b)  to  compare  AIF  utility  and  utilization  in  replacement  and  continuation  training  units,  and 
(c)  to  provide  a  data  base  that  would  be  useful  both  in  defining  the  requirements  for  future  ATD  procurements  and  in 
developing  subsequent  training  programs  using  ATDs. 


Background/Rationale 

ATDs  are  not  merely  flight  simulators.  They  are  also  equipped  with  sophisticated  hardware  and  software  capabilities 
that  permit  an  instructor  pilot  (IP)  to  control,  monitor,  record,  and  in  some  cases,  fabricate  flight  simulation  training 
missions.  These  capabilities,  collectively  known  as  AIFs,  reflect  the  ATD’s  primary  role  as  a  flight  trainer.  It  is  frequendy 
assumed  that  an  ATD’s  training  value  is  a  function  of  its  fidelity  or  capability  for  simulation,  but  in  reality  the  training 
value  of  an  ATD  is  determined  not  only  by  the  degree  to  which  it  “simulates’’  a  particular  aircraft,  but  also  by  the  way 
in  which  it  is  used  as  an  instructional  device. 

Experimental  evidence  indicates  that  AIF-based  simulator  training  can  be  quite  effective.  However,  AIFs  are  costly 
to  implement,  especially  those  features  that  require  the  development  of  complex  software.  In  order  to  justify  these  costs, 
several  questions  regarding  the  utility  and  utilization  of  AIFs  must  be  answered:  (a)  How  frequently  and  easily  are  AIFs 
used?  (b)  Are  IPs  adequately  training  to  use  AIFs?  (c)  Do  AIFs  have  significant  training  value? 


Approach 

A  survey  on  AIF  use  and  training  value  was  developed  and  administered  to  134  qualified  instructors  assigned  to 
replacement  training  units  (RTUs)  and  continuation  training  units  (CTUs)  at  principal  Tactical  Air  Command  (i.e.,  F-4, 
F-15,  A-10,  and  E-3A)  ATD  sites.  The  survey  was  conducted  at  the  request  of  the  Air  Force  Systems  Command, 
Aeronautical  Systems  Division  (AFSC/ASD). 


Specifics 

Method.  The  survey,  which  was  administered  on-site  to  small  groups  of  instructors  (N  =  2  to  10),  included 
background  information,  a  list  of  17  AIFs  and  their  definitions,  and  five  questions  (seven-point  successive-category  rating 
scales)  regarding  the  utility  and  utilization  of  each  feature. 

Findings! Discussion.  The  results  of  the  survey  indicated  that  most  instructors  receive  littie  training  in  AIF  use  and 
that  most  features  are  not  used  very  often.  Several  factors  appear  to  have  contributed  to  the  low  usage:  (a)  hardware  and/or 
software  unreliability,  (b)  time-consuming  implementation,  (c)  functional  limitations,  and  (d)  design  deficiencies.  The 
results  of  a  multiple  regression  analysis  indicated  that  ease  of  use  and  training  value  accounted  for  most  of  the  variability 
in  the  frequency-of-use  ratings. 

AIF  utility  and  utilization  also  differed  as  a  function  of  training  unit,  this  difference  being  most  apparent  in  the 
F-15  data.  For  example,  F-15  RTU  IPs  received  more  training  in  the  use  of  the  features,  used  them  more  frequently, 
found  them  easier  to  use,  and  rated  them  higher  in  training  value.  Similar  tendencies  were  present  for  the  RTU  units 
at  other  ATD  sites. 

However,  the  relation  between  AIF  utility/utilization  and  the  training  unit  was  not  a  simple  one.  Certain  features 
were  rated  higher  by  RTU  instructors  while  others  were  rated  higher  by  CTU  instructors.  In  most  cases,  these  differences 
reflected  the  contrast  between  the  kinds  of  training  given  in  the  two  training  units.  For  example,  RTU  training  is  primarily 
procedural  in  nature  whereas  CTU  training  places  more  emphasis  on  the  training  of  complete  mission  scenarios. 
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Conchuions/Recommcndations 


The  survey  results  constitute  a  data  base  on  AIF  utility  and  utilization  in  ATDs  that  indicated  most  AIFs  are  not 
very  often  used  because  of  (a)  hardware  and/or  software  unreliability,  (b)  time-consuming  implementation,  (c)  functional 
limitations,  (d)  design  deficiencies,  and  (e)  the  fact  that  most  instructor  pilots  (IPs)  receive  little  training  in  their  use. 
The  perceived  value  of  a  given  ATF  varied  with  the  kind  of  training  (CTU  or  RTU). 

It  is  recommended  that  future  procurement  of  AIFs  be  preceded  by  a  detailed  front  end  analysis  that  clearly  relates 
AIF  capability  to  training  needs.  The  analysis  should  consider  all  known  training  applications  of  the  simulator  as  well 
as  any  major  constraints  in  the  operational  environment.  During  procurement,  AIF  specifications  should  be  prepared 
to  meet  user  needs  and  ensure  equipment  reliability.  After  operational  deployment,  the  user  should  provide  adequate 
instructor/operator  training  in  AIF  use. 


PREFACE 


This  research  was  conducted  to  satisfy  requirements  of  Air  Force  Human 
Resources  Laboratory  Technical  Planning  Objective  3,  the  thrust  of  which  is 
air  combat  tactics  and  training.  The  general  objective  of  this  thrust  is  to 
identify  and  demonstrate  cost-effective  training  strategies  and  training 
equipment  capabilities  for  use  in  developing  and  maintaining  the  combat 
effectiveness  of  Air  Force  aircrew  members.  More  specifically,  the  research 
was  conducted  under  the  Air  Combat  Training  Research  subthrust,  the  goal  of 
which  is  to  provide  a  technology  base  for  training  high  level  and  quickly 
perishable  skills  in  simulated  combat  environments.  Work  Unit  1123-02-34, 
Development  and  Evaluation  of  Advanced  Instructional  Features  addressed  a 
portion  of  this  subthrust.  Capt.  L.  Weikhorst  was  the  project  monitor  and 
Dr.  Donald  J.  Polzella,  under  contract  to  the  University  of  Dayton  Research 
Institute,  was  the  principal  investigator. 

This  effort  was  jointly  coordinated  by  the  Air  Force  Human  Resources 
Laboratory,  Operations  Training  Division,  Williams  AFB,  Arizona;  the  Simulator 
System  Program  Office  (SimSPO)  of  the  Air  Force  Systems  Command,  Aeronautical 
System  Division  (AFSC/ASD),  Wright-Patterson  AFB,  Ohio;  Headquarters  Tactical 
Air  Command  (HQ  TAC/DOT),  Langley  AFB,  Virginia;  and  the  Tactical  Air  Warfare 
Center  (TAWC/TN),  Eglin  AFB,  Florida.  The  assistance  of  Mr.  H.  Craig  McLean 
(ASD/YWE),  Mr.  Nat  Davis  (ASD/YWE),  Mr.  Jim  Brown  (TAWC/TNT),  Maj.  Jim  icenhour 
(HQ  TAC/DOTS),  Maj.  Mark  Nataupsky  (HQ  TAC/DOTSI),  Mr.  Richard  Greatorex 
(AFHRL/OTU),  and  Sgt.  Bernadette  Hill  (AFHRL/OTA)  is  gratefully  acknowledged. 
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AIRCREW  TRAINING  DEVICES:  UTILITY  AND  UTILIZATION 
OF  ADVANCED  INSTRUCTIONAL  FEATURES 
(PHASE  I  -  TACTICAL  AIR  COMMAND) 


INTRODUCTION 


An  Aircrew  Training  Device  (ATO)  Is  a  ground-based  apparatus  that  mimics 
the  performance  and  configuration  of  a  particular  aircraft.  However,  the  ATD 
is  net  merely  a  flight  simulator.  It  is  also  equipped  with  sophisticated 
hardware  and  software  capabilities  that  permit  an  instructor  pilot  (IP)  to 
control,  monitor,  record,  and  in  some  cases,  fabricate  flight  simulation 
training  missions.  These  capabilities,  known  as  advanced  instructional 
features  (AlFs),  reflect  the  primary  role  of  the  ATD  as  a  flight  trainer. 

It  is  frequently  assumed  that  the  training  value  of  an  ATD  is  a  function 
of  its  fidelity  or  capability  for  simulation.  According  to  Adams  (1972),  this 
assumption  could  be  unwarranted: 


I  would  not  consider  the  money  being  spent  on  flight  simu¬ 
lators  as  staggering  if  we  knew  much  about  their  training 
value,  which  we  do  not.  We  build  flight  simulators  as 
realistic  as  possible,  ...  but  the  approach  is  also  a 
cover-up  for  our  ignorance  about  transfer  because  in  our 
doubts  we  have  made  costly  devices  as  realistic  as  we  can 
in  the  hopes  of  gaining  as  much  transfer  as  we  can.  [In 
the  past],  the  users  have  been  willing  to  pay  the  price, 
but  the  result  has  been  an  avoidance  of  the  more  chal¬ 
lenging  question  of  how  the  transfer  might  be  accomplished 
in  other  ways,  or  whether  all  that  complexity  (i.e., 
fidelity)  is  really  necessary,  (pp.  616-617) 


By  providing  AIF  capability,  simulator  manufacturers  apparently  recognize  that 
the  training  value  of  an  ATO  is  determined  not  only  by  the  degree  to  which  it 
faithfully  mimics  a  particular  aircraft  but  also  by  the  way  it  is  used  (see 
Caro,  1973). 

Many  instructional  features  have  already  been  provided  by  ATD  manufacturers, 
and  more  are  being  developed.  They  are  listed  in  Table  1  and  grouped  according 
to  function.  The  list  was  compiled  from  several  sources,  but  it  was  drawn 
primarily  from  the  Semple,  Cotton,  and  Sullivan  (1981)  report  describing  the 
AIF  capability  of  various  military  and  commercial  devices. 

Instructional  features  are  expensive  to  implement,  especially  those 
features  that  require  the  development  of  complex  software,  such  as  automated 
adaptive  training  and  automated  voice  controller.  In  order  to  justify  these 


costs,  some  questions  concerning  the  utility  and  utilization  of  AIFs  must  be 
answered.  How  frequently  are  AIFs  used?  Are  IPs  adequately  trained  to  use 
AIFs?  Do  AIFs  have  significant  training  value? 


PREVIOUS  RESEARCH 

The  answers  to  the  questions  in  the  preceding  paragraph  have  not  been  fully 
provided,  but  relevant  information  is  available.  The  most  complete  discussion 
of  instructional  features  can  be  found  in  the  Semple,  et  al.  (1981)  report. 

This  report  was  based  on  interviews  of  ATD  personnel  at  12  Air  Force,  Navy, 
Army,  Coast  Guard,  and  commercial  training  sites  and  is  one  of  seven  reports 
comprising  the  Air  Force  Simulator  Training  Requirements  and  Effectiveness 
Study  (STRES).  The  report  describes  over  20  features  and  discusses  each  in 
terms  of  its  operation,  related  features,  instructional  value,  observed 
applications,  utility  (use-related)  information,  related  research,  and  design 
considerations.  The  interviews  were  "guided"  by  a  checklist  of  topics,  but 
they  were  not  highly  structured.  This  approach  afforded  the  investigators 
flexibility  in  exploring  particular  topics,  but  it  precluded  systematic 
analyses  of  the  data. 

Relevant  information  can  also  be  found  in  several  reports  describing  the 
automated  instructional  system  on  the  Advanced  Simulator  for  Pilot  Training 
(ASPT)  located  at  Williams  AFB  (Faconti,  Mortimer,  &  Simpson,  1970;  Knoop, 

1973;  Faconti  &  Epps,  1975;  Fuller,  Waag,  &  Martin,  1980).  The  ASPT  is  a 
sophisticated  research  device,  which  incorporates  advanced  visual  and  motion 
systems,  A-10  and  F-16  cockpits,  extensive  AIF  capability,  and  an  automated 
performance  measurement  system.  Notwithstanding  the  apparent  training 
potential  of  the  ASPT,  Gray,  Chun,  Warner,  and  Eubanks  (1981)  found  that  IPs 
tended  to  use  the  device  in  a  fairly  conventional  manner.  With  few  exceptions, 
the  instructional  features  were  rarely  used. 

AIF  utility  information  is  available  in  an  important  series  of  reports  by 
R.  G.  Hughes  et  al.  (Bailey  &  Hughes,  1980;  Bailey,  Hughes,  &  Jones,  1980; 
Hughes,  1979;  Hughes,  Hannon,  &  Jones,  1979;  Hughes,  Lintern,  Wightman,  & 
Brooks,  1981).  The  reports  provide  conceptual  models  for  AIF-based  simulator 
training  programs  and  present  experimental  evidence  aimed  at  determining  the 
training  value  of  particular  features.  It  is  clear  from  these  reports  that 
effective  AIF-based  simulator  training  is  practicable,  but  systematic  analyses 
of  ATD  utility  and  utilization  patterns  are  required  before  optimal  training 
programs,  of  the  kind  envisioned  by  Knoop  (1973),  can  be  designed: 


The  software  which  will  comprise  flight  simulators  of  the 
future  will  consist  primarily  of  sophisticated  advanced 
training  programs  which  automatically  and  optimally  step 
the  student  through  training,  measure  his  performance  at 
each  step,  diagnose  his  problems,  and  alter  the  difficulty 
of  various  tasks  which  are  commensurate  with  his  skill. 

(p.  583) 


Table  1.  Advanced  Instructional  Features 


BRIEFING  FEATURES 

Recorded  Briefing  permits  IP  to  provide  student  with  information 
about  the  simulator  and/or  a  structured  training  mission 
through  audiovisual  media  presentation. 

Demonstration  permits  IP  to  demonstrate  aircraft  maneuvers  by 
prerecording  and  playing  back  a  standardized  segment  of 
simulated  flight. 

Instructor  Pilot  Tutorial  provides  IP  with  self-paced  programmed 
instruction  in  the  capabilities  and  use  of  the  flight 
simulator. 

TRAINING  MANAGEMENT  FEATURES 

Total  System  Freeze  permits  IP  to  suspend  simulated  flight  by 
freezing  all  system  parameters. 

Reset  permits  IP  to  return  the  simulated  aircraft  to  a  stored 
set  of  conditions  and  parameters. 

Trash  and/or  Kill  Override  permits  IP  to  allow  simulated  flight 
to  continue  without  interruption  following  a  "crash"  or 
"kill." 

Automated  Adaptive  Training  is  the  computer-controlled  variation 
in  task  difficulty,  complexity,  and/or  sequence  based  on 
student's  performance. 

Programmed  Mission  Scenarios  are  computer -control led  standardized 
training  missions  based  on  pre-programmed  event  sequences. 
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Table  1.  Advanced  Instructional  Features  (continued) 


VARIATION  OF  TASK  DIFFICULTY  FEATURES 

Automated  Malfunction  Insertion  permits  IP  to  preprogram  a 
sequence  of  aircraft  component  malfunctions  and/or 
emergency  conditions. 

Environmental  permits  IP  to  vary  environmental  conditions  such 
as  wind  direction  and  velocity,  turbulence,  temperature, 
and  visibility. 

Oynami cs  permits  IP  to  vary  flight  dynamics  characteristics 
such  as  stability,  system  gain,  cross-coupling,  etc. 

Motion  permits  IP  to  provide  student  with  platform 

motion  system  cues  such  as  roll,  pitch,  lateral,  and 
vertical. 

Flight  System  Freeze  permits  IP  to  simultaneously  freeze  flight 
control  and  propulsion  systems,  position,  altitude,  and 
heading. 

Position  Freeze  permits  IP  to  simultaneously  freeze  latitude 
and  longitude. 

Attitude  Freeze  permits  IP  to  simultaneously  freeze  pitch,  bank, 
and  heading. 

Parameter  Freeze  permits  IP  to  freeze  any  one  or  a  combination 
of  flight  parameters. 
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Table  1.  Advanced  Instructional  Features  (concluded) 


INSTRUCTOR  MONITOR  AND  FEEDBACK  FEATURES 

Closed  Circuit  TV  permits  IP  to  monitor  student's  behavior  from 
the  instructor  console. 

Repeaters/Annunci ators  provide  IP  with  replicas  or  analog 

representations  of  flight  instruments  and  controls  at  the 
instructor  console. 

Instructor  Console  Displays  permit  IP  to  view  alphanumeric 
and/or  graphic  CRT  displays  of  performance  data  at  the 
instructor  console. 

Automated  Performance  Alert  provides  IP  with  visual  and/or 
auditory  signals  that  indicate  specific  performance 
deficiencies. 

STUDENT  FEEDBACK  FEATURES 

Record/Playback  permits  IP  to  record  and  subsequently  play  back 
all  events  that  occurred  during  a  segment  of  simulated 
flight. 

Automated  Performance  Feedback  provides  student  with  visual 
and/or  auditory  signals  (including  verbal  messages)  that 
identify  performance  deficiencies. 

Automated  Voice  Controller  is  the  computer-based  technology  that 
simulates  the  role  of  controller  by  combining  speech 
generation,  speech  recognition,  and  situation  awareness 
capabilities. 

Hard  Copy  provides  a  record  of  alphanumeric  and/or  graphic 

performance  data  from  the  automated  performance  measurement 
system. 
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THE  PRESENT  INVESTIGATION 


The  present  investigation  was  conducted  at  the  request  of  the  Simulator 
System  Program  Office  (SimSPO)  of  the  Air  Force  Systems  Command  Aeronautical 
Systems  Division,  (AFSC/ASO).  The  objectives  of  the  investigation  were: 

1.  To  document  and  compare  the  utilization  (i.e.,  ease  and  frequency  of 
use)  of  AIFs. 

2.  To  document  and  compare  the  utility  (i.e.,  training  value)  of  AIFs. 

3.  To  compare  the  utility  and  utilization  patterns  of  AIFs  in  replacement 
and  continuation  training  units. 

A  broader  objective  of  the  investigation  is  to  provide  a  data  base  that  would 
be  helpful  both  in  defining  the  requirements  for  future  ATD  procurements  and 
in  developing  subsequent  ATD  training  programs. 

The  objectives  will  be  accomplished  in  two  phases  by  means  of  a  survey  of 
flight  instructors  from  the  Air  Force  Major  Commands  (MAJCOMS).  Phase  I  is 
described  in  this  report  and  includes  instructors  from  principal  Tactical  Air 
Command  (TAC)  ATD  sites  (F-4,  F-15,  A-10,  and  E-3A  aircraft).  Phase  II  will 
include  instructors  from  principal  Strategic  Air  Command  (SAC),  Military 
Airlift  Command  (MAC),  and  Air  Training  Command  (ATC)  ATD  sites  (FB-111, 

C-130,  C-141,  C-5,  CH-3,  HH-53,  T-37,  and  T-38  aircraft). 


METHOD 


Subjects 

The  subjects  in  Phase  1  were  134  simulator-qualified  IPs  and  Weapons 
Director  Instructors  (WDIs)  assigned  to  replacement  training  units  (RTUs, 
N=84)  or  continuation  training  units  (CTUs,  N=50)  at  the  following  TAC  bases: 
George  AFB  (F-4E,  F-4G;  RTU,  CTU),  Luke  AFB  (F-15,  RTU),  Langley  AFB  (F-15, 
CTU),  Eglin  AFB  (F-15,  CTU),  Davis-Monthan  AFB  (A-10,  RTU),  Myrtle  Beach  AFB 
(A-10,  CTU),  and  Tinker  AFB  (E-3A;  RTU,  CTU).  The  number  of  IPs  and  WDIs 
surveyed  at  each  site  is  shown  in  Table  2. 

Questionnaire 

The  questionnaire  that  was  used  to  survey  the  instructors  is  shown  in 
Appendix  A.  It  includes  background  information  (e.g.,  flying  and  simulator 
experience),  a  brief  description  of  a  typical  simulator  training  session,  a 
list  of  17  AIFs  and  their  definitions,  and  five  questions  concerning  the 
utility  and  utilization  of  each  feature: 

1.  How  often  have  you  used  it? 

2.  How  easy  is  it  to  use? 

3.  How  much  training  did  you  receive  in  its  use? 

4.  What  is  its  training  value? 

5.  What  is  its  potential  training  value? 


i 


I 


i 


Table  2.  The  Number  of  Instructors  Surveyed  at  Each  ATD  Site 


Training  Unit 
RTU 


ATD  Site 


CTU 


For  the  fifth  question,  instructors  were  asked  to  assume  that  they  had  no 
prior  knowledge  of  the  features  and  to  base  their  answers  on  the  feature 
definitions  alone.  This  question  was  included  in  order  to  achieve  a  common 
basis  for  comparison  among  all  instructors.  This  was  not  otherwise  possible 
because  the  various  ATDs  were  not  similarly  equipped. 

Responses  to  each  question  were  indicated  by  checking  the  appropriate 
interval  along  a  seven-point  successive-category  scale.  (A  zero-point 
interval  was  included  for  indicating  that  a  particular  feature  was  never  used 
or  was  unavailable.  Subjects  were  instructed  to  "consider  a  feature  to  be 
available  as  long  as  it  is  incorporated  on  the  simulator  and  regardless  of 
whether  it  is  fully  operational.")  In  order  to  facilitate  responding,  the 
intervals  of  each  scale  were  labeled  with  descriptive  adjectives.  Additional 
space  was  provided  for  comments. 

Procedure 


The  questionnaire  was  administered  on-site  to  small  groups  of  instructors 
(2  to  10).  The  instructors  were  briefed  on  the  purpose  of  the  investigation 
and  copies  of  the  questionnaire  were  distributed  and  thoroughly  reviewed  prior 
to  being  filled  out.  The  questionnaire  could  be  completed  in  approximately  20 
minutes. 


RESULTS 


The  raw  data  included  the  instructors'  flying  and  simulator  experience  in 
number  of  hours,  their  descriptions  of  a  typical  training  session,  and  their 
responses  to  each  question,  coded  as  0  to  7.  (Except  as  noted,  responses  of 
0,  which  indicated  that  a  particular  feature  was  unavailable  or  never  used, 
were  not  included  in  the  analyses.) 

The  raw  data  were  classified  by  ATO  (F-4,  F-15,  A-10,  E-3A  Flight 
Simulator,  E-3A  Mission  Simulator),  training  (RTU,  CTU),  question  (1  through 
5),  and  AIF  (1  through  17).  Due  to  the  unequal  number  of  instructors  and  the 
different  f eatures-capability  at  each  site  (see  Appendix  B),  the  data  matrix 
was  unbalanced.  Therefore,  the  data  for  each  ATD  were  analyzed  separately. 

The  data  were  then  pooled  and  reanalyzed  in  order  to  assess  the  general 
pattern  of  results.  All  analyses  were  computed  on  a  Univac  1100/81  system 
using  BMDP  statistical  software  (Dixon,  1981). 

Sample  Size 

A  sample  of  134  instructors  (84  RTU  and  50  CTU)  were  surveyed  in  this 
study.  In  analyzing  the  data,  this  sample  was  treated  as  though  it  had  been 
randomly  drawn  from  an  infinite  population  of  flight  instructors.  In  fact, 
the  sample  was  drawn  from  a  finite  population  and,  moreover,  constituted  a 
large  proportion  of  that  population.  Thus,  the  parametric  analyses  described 
in  this  section  (ANOVAs,  t-tests)  are  conservatively  biased,  and  the  resulting 
inferences  are  made  with  considerable  confidence. 
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Instructors'  Flying  and  Simulator  Experience 


The  instructors'  flying  and  simulator  experience  is  summarized  for  each 
ATD  site  in  Tables  3  to  7  and  for  the  pooled  data  in  Table  8.  The  RTU-CTU 
means  for  each  type  of  experience  were  compared  using  t-tests.  The 
significant  comparisons  are  indicated  on  the  tables  with  asterisks. 

The  RTU  and  CTU  means  were  statistically  equivalent  at  the  F-4  and  E-3A 
sites.  However,  there  were  several  significant  differences  at  the  other 
sites.  Table  8  shows  that,  overall,  RTU  instructors  reported  significantly 
more  simulator  flying  time  and  simulator  instruction  time,  t(126)  *  2.44, 
p  <  .05  and  t(128)  =  2.17,  £  <  .05,  respectively.  There  were  no  significant 
overall  differences  in  aircraft  time. 

The  ''Typical1'  Training  Session 

The  typical  simulator  training  session  varies  by  ATD  and  by  type  of 
training  unit.  For  example,  a  typical  F-4  RTU  session  consists  primarily  of 
preflight,  inflight,  and  postflight  procedural  training  with  particular 
emphasis  on  instrument  and  emergency  procedures.  The  session  may  also  include 
practice  in  certain  tactical  skills  such  as  intercepts,  air-to-ground  attack, 
and  the  use  of  radar  warning  systems.  In  contrast,  tactical  skills  are 
emphasized  in  the  typical  F-4  CTU  session,  which  frequently  centers  around  a 
highly  structured  tactical  scenario.  The  F-15  RTU  and  CTU  training  sessions 
are  similar  to  those  of  the  F-4. 

The  A-10  RTU  and  CTU  training  sessions  do  not  markedly  differ.  In  both 
cases,  the  emphasis  is  on  procedural  training.  Apparently,  this  emphasis  is 
one  of  necessity.  The  RTU  ATD  ( Davis -Monthan  AFB)  has  only  a  dusk/night 
visual  system,  while  the  CTU  ATD  (Myrtle  Beach  AFB)  has  no  visual  system  at 
all.  Without  a  sophisticated  visual  system,  neither  ATD  can  effectively 
simulate  the  principal  A-10  tactical  capability,  air-to-ground  attack. 

The  size  and  functional  characteristics  of  the  typical  Airborne  Warning 
and  Control  System  (AWACS)  crew  necessitates  the  use  of  two  E-3A  ATDs.  E-3A 
flight  training  is  carried  out  by  IPs  in  the  Flight  Simulator,  while  E-3A 
aircrew  mission  training  is  carried  out  by  WDIs  in  the  Mission  Simulator. 
Training  sessions  on  either  ATD  may  last  several  hours. 

RTU  and  CTU  Flight  Simulator  sessions  are  similar  and  consist  primarily  of 
emergency  procedural  training.  E-3A  Mission  Simulator  sessions  are,  as  one 
WDI  described  them,  "designed  to  train  students  within  a  high  density, 
multiple  event  scenario."  Both  RTU  and  CTU  sessions  emphasize  crew 
coordination,  procedural  training,  and  fighter  intercept  monitoring  and 
control'.  CTU  sessions  are  typically  based  on  formalized  scenarios  (Red  Flag, 
Blue  Flag),  and  CTU  students  may  be  required  to  "play"  various  roles,  such  as 
pilot,  ground  monitor,  senior  weapons  director. 

Student  preflight  and  postflight  briefings  are  common  across  all  ATDs. 
However,  they  occur  more  often  at  RTU  training  sites. 
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Table  3.  F-4  IPs:  Experience  in  Mean  Number  of  Hours 


RTU" 

CTU 

Aircraft 

Flying  experience 

1152.4 

1090.0 

IP  experience 

382.5 

282.5 

Simulator 

Flying  experience 

43/. 6 

330.0 

IP  experience 

242.2 

171.5 

Table  4.  F-15  IPs:  Experience  in  Mean 

Number  of  Hours 

RTU 

CTU 

Aircraft 

Flying  experience 

747.5 

694.2 

*  IP  experience 

287.5 

153.7 

Simulator 

Flying  experience 

229.5 

206.6 

IP  experience 

171.2 

115.3 

*  p  <  .025. 
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Table  5.  A-10  IPs:  Experience  in  Mean  Number  of  Hours 


RTU 

CTU 

Aircraft 

Flying  experience 

645.4 

503.8 

*  IP  experience 

256.9 

56.7 

Simul ator 

Flying  experience 

27.9 

26.2 

IP  experience 

21.9 

21.2 

*  p  <  .01. 


Table  6.  E-3A  IPs:  Experience  in  Mean  Number  of  Hours 


RFU 

CTU 

Aircraft 

Flying  experience 

1570.0 

1266.7 

IP  experience 

382.0 

260. 0 

Simulator 

Flying  experience 

516.0 

466.0 

IP  experience 

362.4 

332.7 

' 
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Table  7.  E-3A  WDIs:  Experience  in  Mean  Number  of  Hours 


RTU 

CTU 

Aircraft 

Flying  experience 

1069.1 

838.3 

WDI  experience 

400.6 

212.5 

Simul ator 

*  Flying  experience 

1244.1 

591.7 

WDI  experience 

529.4 

217.7 

*  p  <  .05. 


Table  8.  Pooled  Data:  Instructors' 
in  Mean  Number  of  Hours 

Experience 

RTU 

CTU 

Aircraft 

Flying  experience 

907.1 

779.3 

IP  or  WDI  experience 

324.6 

169.6 

Simulator 

*  Flying  experience 

429.1 

249.8 

*  IP  or  WDI  experience 

222.4 

129.3 

Ratings  of  the  frequency  of  A1F  use  are  summarized  for  each  ATD  site  in 
Tables  9  to  13  and  for  the  pooled  data  in  Table  14.  The  individual  ratings 
ranged  from  1  (never  use)  to  7  (use  most  often).  (Note:  Ratings  of  0,  i.  e., 
unavailable,  were  transformed  to  ratings  of  1,  i.e.,  never  use,  before  the 
data  were  analyzed.  The  tabled  means  incorporate  these  transformed  ratings.) 

The  frequency  of  A1F  use  appears  to  be  generally  low.  Except  for  the  E-3A 
IP  data,  most  of  the  means  range  from  1.5  to  3.5  (i.e.,  rarely  to 
occasionally)  with  an  overall  mean  of  2.8.  This  is  partly  due  to  the  high 
percentage  of  instructors  who  indicated  that  particular  features  were 
unavailable  for  use.  This  percentage  was  25. 7%  overall  and  was  as  high  as  76% 
for  recorded  briefing. 


A  two-factor  (Training  x  Available  AlFs)  repeated  measures  analysis  of 
variance  (for  unequal  N)  was  used  to  analyze  the  data  from  each  ATD.  A 
significant  main  effect  of  A1F  was  found  in  every  case,  £  <  .001,  indicating 
that  the  various  AlFs  are  not  used  equally  often.  In  most  cases,  reset, 
environmental,  the  various  freezes,  and  crash  override,  if  available,  were 
used  significantly  more  often  than  recorded  briefing, "demonstration, 
record/playback,  hard  copy,  automated  adaptive  training,  and  programmed 
mission  scenarios.  In  contrast,  demonstration,  hard  copy,  and  programmed 
mission  scenarios  were  among  the  features  used  most  often  on  the  E-3A  Mission 
Simulator.  (Note:  these  differences  were  determined  by  the  Dunn  test,  Keppel, 
1973,  pp.  147-149.) 

The  main  effect  of  training  was  significant  for  the  F-15  ATD,  F(l,37)  = 
6.41,  £  <  .025,  and  for  the  A-10  ATD,  F( 1, 36)  *  4.29,  £  <  .05.  In  both 
cases,  the  RTU  ratings  were  significantly  higher  than  the  CTU  ratings. 

The  interaction  of  training  and  AIF  was  significant  for  the  F-15  ATD, 

F( 8,296)  «  8.61,  £  <  .001,  the  A-10  ATD,  F(12,432)  =  3.21,  £  <  .01,  and  the 
E-3A  flight  simulator,  F(9,54)  =  4.71,  £  <  .001.  A  significant  interaction 
implies  that  the  pattern  of  AIF  use  is  different  for  the  two  training  groups. 
In  order  to  determine  the  locus  of  this  interaction,  the  Dunn  test  was  used  to 
make  post -hoc  comparisons  between  the  various  RTU/CTU  mean  pairs.  The 
significant  differences  are  indicated  in  Tables  10,  11,  and  12  by  asterisks. 
For  example.  Table  10  shows  that  reset  and  flight  system  freeze  are  used  more 
frequently  by  F-15  RTU  IPs,  £  <  .01,  while  programmed  mission  scenarios  are 
used  more  frequently  by  F-15  CTU  IPs,  £  <  .01. 


Table  9.  F-4  Simulator:  Mean  Rated  Frequency  of  AIF  Use 


TO  O  O  O 


feature 

RTU 

CTU 

Mean 

Reset 

5.2 

5.0 

5.1 

Environmental 

4.1 

3.8 

Auto  Malfunction  Insertion 

1.4 

1.0 

1.2 

Flight  System  Freeze 

3.5 

3.5 

3.5 

Position  Freeze 

3.1 

3.3 

3.2 

Parameter  Freeze 

2.8 

3.4 

3.0 

Crash  Override 

6.4 

6.5 

6.4 

+  Auto  Performance  Feedback 

1.5 

1.7 

1.5 

Hard  Copy 

1.8 

1.8 

1.8 

+  Auto  Adaptive  Training 

1.4 

2.7 

1.6 

Programmed  Mission  Scenarios 

2.1 

3.3 

2.5 

Grand  Mean 

3.1 

3.4 

3.2 

+  Note:  These  AIFs  are  available 

only  on 

the  F-4E 

simulator. 
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Reset 

4.8 

2.8 

3.9 

Environmental 

4.2 

3.1 

3.7 

Auto  Malfunction  insertion 

3.0 

3.7 

3.4 

Flight  System  Freeze 

6.3 

3.9 

5.1 

Crash  Override 

5.6 

5.1 

5.3 

Motion 

2.5 

1.5 

2.0 

Hard  Copy 

1.2 

2.1 

1.6 

Programmed  Mission  Scenarios 

2.1 

4J, 

hi 

Grand  Mean 

3.7 

3.3 

3.5 

*  p  <  .01. 
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Table  11.  A-10  Simulator:  Mean  Rated  Frequency  of  AIF  Use 


:eature . 

RTU 

CTU 

Mean 

Reset 

5.0 

4.3 

4.8 

Total  System  Freeze 

4.5 

3.0 

4.0 

Recorded  Briefing 

1.2 

1.0 

1.1 

Demonstration 

1.8 

1.2 

1.6 

Record/Playback 

1.6 

1.3 

1.5 

Environmental 

4.5 

3.0 

4.0 

Auto  Malfunction  Insertion 

2.6 

2.6 

2.6 

r  Flight  System  Freeze 

1.9 

3.5 

2.4 

Position  Freeze 

3.0 

2.3 

2.8 

Parameter  Freeze 

2.9 

2.0 

2.6 

Crash  Override 

4.6 

3.8 

4.3 

Hard  Copy 

2.6 

1.6 

2.3 

Programmed  Mission  Scenarios 

1.0 

1.5 

1.2 

Grand  Mean 

2.9 

2.4 

2.7 

Table  12.  E-3A  Flight  Simulator:  Mean  Rated  Frequency  of  A1F  Use  (IPs) 


Feature 

RTU 

CTU 

Mean 

Reset 

6.2 

4.7 

5.6 

Environmental 

6.0 

6.3 

6.1 

Auto  Malfunction  Insertion 

5.4 

4.3 

5.0 

Flight  System  Freeze 

6.4 

4.0 

5.5 

Position  Freeze 

6.4 

5.3 

6.0 

Parameter  Freeze 

4.8 

5.0 

4.9 

Crash  Override 

4.6 

5.0 

4.8 

Motion 

6.4 

7.0 

6.6 

Hard  Copy 

1.0 

1.0 

1.0 

*  Programmed  Mission  Scenarios 

1.4 

6.0 

hi 

Grand  Mean  4.9 


4.9 


4.9 


Table  13.  E-3A  Mission  Simulator:  Mean  Rated 
Frequency  of  AIF  Use  (WDls) 


Table  14.  Pooled  Data:  Mean  Rated  Frequency  of  A1F  Use 


Feature 

RTU 

CTU 

Mean 

**  Reset 

5.1 

3.9 

4.6 

Total  System  Freeze 

3.2 

2.4 

3.0 

Recorded  Briefing 

1.3 

1.3 

1.3 

*  Demonstration 

2.4 

1.8 

2.2 

Record/Playback 

1.4 

1.3 

1.4 

Environmental 

3.9 

3.3 

3.7 

Auto  Malfunction  Insertion 

2.7 

2.8 

2.7 

Flight  System  Freeze 

3.9 

3.7 

3.8 

Position  Freeze 

3.4 

3.1 

3.3 

Parameter  Freeze 

3.1 

2.9 

3.0 

Crash  Override 

4.9 

4.8 

4.8 

Motion 

3.3 

2.3 

2.8 

Auto  Performance  Feedback 

2.5 

2.3 

2.5 

Hard  Copy 

2.0 

1.9 

2.0 

*  Auto  Adaptive  Training 

1.4 

2.7 

1.6 

**  Programmed  Mission  Scenarios 

2.0 

3.3 

2.5 

Unweighted  Grand  Mean 

2.9 

2.7 

2.8 

*  p  <  .05. 

**  p  <  .001. 
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The  pooled  data,  summarized  in  Table  14,  must  be  interpreted  cautiously, 
since  the  pooled  data  matrix  was  unbalanced.  The  nonparametric  Wilcoxen  test 
was  used  to  compare  the  RTU  and  CTU  ratings  overall,  while  t-tests  were  used 
to  make  separate  RTU-CTU  comparisons  for  each  feature.  The  significant 
differences  are  indicated  in  Table  14  with  asterisks.  Thus,  reset  and 
demonstration  are  used  more  frequently  by  RTU  instructors,  t( 70)  =  3.75, 

2  <  .001  and  t ( 97 )  =  2.47,  £  <  .025,  respectively,  while  auto  adaptive 
training  and  programmed  mission  scenarios  are  used  more  frequently  by  CTU 
instructors,  t(15)  =  -  2.14,  £  <  .05  and  t(87)  =  -  4.23,  £  <  .001, 
respecti vely.  There  was  no  overall  difference  between  the  two  groups' 
ratings,  T  =  31.5,  £  >  .05. 

Ease  of  A1F  Use 

Ratings  of  the  ease  of  A1F  use  are  summarized  for  each  ATD  site  in  Tables 
15  to  19  and  for  the  pooled  data  in  Table  20.  The  individual  ratings  ranged 
from  1  (most  difficult)  to  7  (easiest).  Most  of  the  means  are  in  the  4.0  to 
6.0  range  suggesting  that  most  AlFs  are  at  least  moderately  easy  to  use.  The 
features  were  rated  similarly  at  each  site,  with  certain  exceptions.  For 
example,  reset,  environmental .  the  various  freezes,  motion,  and  crash  override 
(those  features  used  most  frequently)  are  apparently  easy  to  use  wherever 
available.  On  the  other  hand,  record/playback.  auto  performance  feedback, 
hard  copy,  and  programmed  mission  scenarios  were  rated  differently  across  some 
of  the  sites.  For  example,  A-10  IPs  rated  record/playback  as  moderately  easy 
to  use,  whereas  E-3A  WOls  rated  it  as  very  difficult.  F-4  IPs  rated  auto 
performance  feedback  as  difficult  to  use,  whereas  E-3A  WDls  rated  it  as  very 
easy.  Finally,  hard  copy  and  programmed  mission  scenarios  are  moderately  easy 
to  implement  on  all  ATOs  except  the  E-3A  mission  simulator  (WDIs)  and  A-10 
simulator,  respectively. 

Unlike  the  frequency  of  use  data,  the  means  in  Tables  15  to  20  are  based 
on  a  variable  frequency  that  reflects  the  number  of  instructors  who  actually 
used  each  feature.  Thus,  all  the  data  matrices  were  unbalanced.  Consequently, 
for  each  ATD  and  for  the  pooled  data,  the  nonparametric  Wilcoxen  test  was  used 
to  compare  the  RTU  and  CTU  ratings,  overall,  and  t-tests  were  used  to  make 
separate  RTU-CTU  comparisons  for  each  feature. 

The  overall  comparison  was  significant  only  for  the  F-4  and  F-15  data. 

The  F-4  CTU  IPs  tended  to  give  higher  ratings  than  did  the  F-4  RTU  IPs, 

T  =  1.5,  £  <  .01,  while  F-15  RTU  IPs  gave  higher  ratings  than  did  F-15  CTU 
IPs,  T  =  0,  £  <  .01.  The  significant  comparisons  for  each  feature  are 
indicated  in  Tables  15  and  16  by  asterisks.  Thus,  parameter  freeze  was  rated 
higher  by  F-4  CTU  IPs  (Table  15),  t(17)  =  -2.54,  p  <  .025,  while  environmen- 
tal,  fliqht  system  freeze,  and  crash  override  were  rated  hiqher  by  F-15  RTU 
TPs  (table  16),  t(35)  =  3.00,  £  <  .01,  t( 15)  =  2.42,  £  <  .05,  t(36)  =  3.51, 

£  <  .01,  respectively.  Although  the  pooled  data  revealed  no  overall  differ¬ 
ence  between  the  RTU  and  CTU  ratings,  T  =  41.5,  £  >  .05,  fliqht  system  freeze 
was  rated  easier  to  use  by  RTU  instructors,  t(78)  =  2.19,  p  <  .05. 


Table  15.  F-4  Simulator:  Mean  Rated  Ease  of  A1F  Use 


'-'A'-'- 


_ .  >  ' 


Feature 

RTU 

CTU 

Mean 

ifl 

Reset 

5.9 

6.0 

5.9 

Environmental 

5.1 

5.3 

5.2 

£ 

Auto  Malfunction  Insertion 

3.0 

5.0 

3.3 

i 

Flight  System  Freeze 

5.9 

5.8 

5.9 

V: 

f.. 

Position  Freeze 

5.2 

5.4 

5.3 

*  Parameter  Freeze 

5.0 

6.4 

5.5 

$ 

Crash  Override 

6.5 

6.5 

6.5 

--S 

+  Auto  Performance  Feedback 

2.1 

3.0 

2.2 

Hard  Copy 

3.3 

3.5 

3.4 

i 

+  Auto  Adaptive  Training 

2.4 

3.5 

2.7 

•  * 

w* 

Programmed  Mission  Scenarios 

2.9 

4.1 

M 

s- 

1  4 

c 

Unweighted  Grand  Mean 

4.3 

5.0 

4.5 

Table  16.  F-15  Simulator:  Mean  Rated  Ease  of  A1F  Use 


Feature 

RTU 

CTU 

Mean 

Reset 

5.3 

4.7 

5.1 

**  Environmental 

5.4 

4.1 

4.8 

Auto  Malfunction  Insertion 

4.9 

4.4 

4.6 

*  Flight  System  Freeze 

6.9 

6.3 

6.7 

**  Crash  Override 

6.7 

5.8 

6.3 

Motion 

5.4 

4.4 

4.9 

Hard  Copy 

4.8 

4.1 

4.3 

Programmed  Mission  Scenarios 

4.9 

hi 

4.8 

Programmed  Mission  Scenarios 
Unweighted  Grand  Mean 


H 

si 


5.5 


.8 


5.2 


if:  Table  17.  A-10  Simulator:  Mean  Rated  Ease  of  A1F  Use 


Feature 

~ rTu 

CTU 

Mean 

Reset 

5.1 

5.4 

5.2 

Total  System  Freeze 

5.7 

5.8 

5.7 

Recorded  Briefing 

5.0 

- - 

5.0 

Table  18.  E-3A  Flight  Simulator:  Mean  Rated  Ease  of  A1F  Use  (IPs) 


:eature 

RTU 

CTU 

Mean 

Reset 

6.4 

5.7 

6.1 

Environmental 

5.4 

6.0 

5.6 

Auto  Malfunction  insertion 

4.2 

4.0 

4.1 

Flight  System  Freeze 

5.8 

5.3 

5.6 

Position  Freeze 

6.0 

5.3 

5.8 

Parameter  Freeze 

5.5 

6.3 

5.9 

Crash  Override 

5.8 

5.3 

5.6 

Motion 

6.2 

6.3 

6.3 

Hard  Copy 

2.0 

— 

2.0 

Programmed  Mission  Scenarios 

3.0 

5.0 

4^3 

Unweighted  Grand  Mean 

5.0 

5.5 

5.1 

Table  19.  E-3A  Mission  Simulator:  Mean  Rated  Ease  of  AIF  Use  (WDIs) 


feature 

RTU 

CTU 

Mean 

Total  System  Freeze 

4.7 

2.7 

4.0 

Recorded  Briefing 

4.6 

3.5 

4.3 

Demonstration 

4.8 

4.0 

4.6 

Record/Playback 

2.3 

1.5 

2.1 

Environmental 

4.6 

4.5 

4.6 

Crash  Override 

5.8 

6.0 

5.9 

Automated  Performance  Feedback 

6.3 

6.0 

6.2 

Hard  Copy 

5.6 

5.6 

5.6 

Programmed  Mission  Scenarios 

3.7 

4.2 

3.8 

Unweighted  Grand  Mean 

4.7 

4.2 

4.6 
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Table  20.  Pooled  Data:  Mean  Rated  Ease  of  A1F  Use 


Feature 

RTU 

CTU 

Mean 

Reset 

5.4 

5.4 

5.4 

Total  System  Freeze 

5.8 

5.6 

5.7 

Recorded  Briefing 

4.6 

4.0 

4.4 

Demonstration 

4.1 

4.3 

4.2 

Record/Playback 

3.1 

3.6 

3.2 

Environmental 

4.8 

4.7 

4.8 

Auto  Malfunction  Insertion 

4.2 

3.9 

4.1 

*  Flight  System  Freeze 

6.2 

5.7 

6.0 

Position  Freeze 

5.2 

5.2 

5.2 

Parameter  Freeze 

5.0 

5.6 

5.2 

Crash  Override 

6.2 

5.9 

6.1 

Motion 

5.5 

4.8 

5.2 

Auto  Performance  Feedback 

4.4 

4.5 

4.5 

Hard  Copy 

4.5 

4.4 

4.4 

Auto  Adaptive  Training 

2.5 

3.5 

2.7 

Programmed  Mission  Scenarios 

3.8 

4.3 

Unweighted  Grand  Mean 

4.7 

4.7 

4.7 

*  n  <  .05. 


Training  in  AIF  Use 

The  amount  of  training  received  in  A1F  use  is  summarized  for  each  ATD  site 
in  Tables  21  to  25  and  for  the  pooled  data  in  Table  26.  The  individual  ratings 
ranged  from  1  (none)  to  7  (greatest).  As  was  the  case  for  the  frequency-of- 
use  data,  ratings  of  0  were  transformed  to  ratings  of  1  before  the  data  were 
analyzed.  The  tabled  means  incorporate  these  transformed  ratings. 

Except  for  the  E-3A  IP  data,  most  of  the  means  range  from  1.5  to  4.5 
(i.e.,  minimal  or  moderate)  and  are  similar  to  those  obtained  from  the 
f requency-of-use  data.  It  appears  that  most  instructors  do  not  receive 
extensive  training  in  A1F  use. 

A  two-factor  (Training  x  Available  AlFs)  repeated  measures  analysis  of 
variance  (for  unequal  N)  was  used  to  analyze  the  data  from  each  ATO.  The  main 
effect  of  AIF  was  significant  in  every  case,  £  <  .001.  The  differences 
between  features,  determined  by  the  Dunn  test,  were  similar  to  those  revealed 
in  the  frequency  of  use  data.  In  most  cases,  reset,  environmental,  the 
various  freezes,  and  crash  override,  the  same  features  that  were  frequently 
used,  received  significantly  more  training  than  did  infrequently  used 
features,  such  as  recorded  briefing,  demonstration,  record/playback,  automated 
adaptive  training,  and  programmed  mission  scenarios. 

The  main  effect  of  training  was  significant  only  for  F-15  IPs,  F(l,37)  = 
5.90,  £  <  .025.  F-15  RTU  IPs  received  more  training  than  did  F-15  CTU  IPs. 

(This  effect  was  also  present  in  the  frequency  of  use  data.)  In  most  cases, 
then,  RTU  and  CTU  instructors  received  comparable  amounts  of  training  overall. 

The  interaction  of  training  and  AIF  was  significant  for  the  F-15  ATD, 

£(8,296)  =  6.63,  £  <  .001,  and  the  A-10  ATD,  £(12,432)  =  2.12,  £  <  .05.  In 

the  case  of  the  F-15  data,  RTU  IPs  received  more  training  in  the  use  of  reset, 
environmental,  and  flight  system  freeze,  whereas  CTU  IPs  received  more  training 
in  the  use  of  hard  copy.  In  the  case  of  the  A-10  data,  the  locus  of  the 
interaction  was  not  revealed  by  post-hoc  comparisons.  However,  the  relative 
differences  between  the  means  are  similar  to  those  in  the  A-10  f requency-of- 
use  data.  (Compare  Tables  11  and  23.) 

The  analysis  of  the  pooled  data  revealed  no  overall  difference  between  the 

RTU  and  CTU  ratings,  T  -  47.5,  £  >  .05.  However,  RTU  instructors  received 

more  training  in  reset,  t(76)  =  3.08,  £  <  .01,  whereas  CTU  instructors 
received  more  training  in  automated  adaptive  training  and  programmed  mission 
scenarios,  t(15)  =  -  2.29,  £  <  .05  and  t ( 132 )  =  -  2^23,  £  <  .05, 
respectively.  (Note:  automated  adaptive  training  was  not  available  on  most 


ATDs.  Therefore,  this  difference  may  not  generalize.) 
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Table  21.  F-4  Simulator:  Mean  Amount  of  Training  Received  in  A1F  Use 


Feature 

RTU 

CTU 

Mean 

Reset 

3.7 

4.0 

3.8 

Environmental 

2.8 

3.2 

2.9 

Auto  Malfunction  Insertion 

1.2 

1.2 

1.2 

Flight  System  Freeze 

3.1 

3.3 

3.2 

Position  Freeze 

3.2 

3.8 

3.5 

Parameter  Freeze 

2.6 

2.8 

2.7 

Crash  Override 

3.2 

3.8 

3.5 

+  Auto  Performance  Feedback 

1.6 

1.3 

1.6 

Hard  Copy 

2.0 

1.8 

1.9 

+  Auto  Adaptive  Training 

1.3 

2.7 

1.5 

Programed  Mission  Scenarios 

L8 

2.6 

2.1 

Grand  Mean 

2.4 

2.9 

2.6 

+  Note:  These  AIFs  are  available  only 

on  the  F 

:-4E  simulator. 

! 


1 


Table  22.  F-15  Simulator:  Mean  Amount  of  Training  Received  in  A1F  Use 


feature 

RTU 

CTU 

Mean 

**  Reset 

4.0 

1.9 

3.0 

*  Environmental 

4.0 

2.9 

3.5 

Auto  Malfunction  Insertion 

3.2 

3.6 

3.4 

*  Flight  System  Freeze 

4.4 

3.2 

3.8 

Crash  Override 

4.3 

3.8 

4.1 

Motion 

3.4 

2.4 

2.9 

*  Hard  Copy 

1.3 

2.5 

1.9 

Programmed  Mission  Scenarios 

3.0 

2JS 

hi 

Grand  Mean 

3.4 

2.9 

3.2 

*  p  <  .05. 

**  p  <  .01. 


Table  23.  A-10  Simulator:  Mean  Amount  of  Training  Received  in  A1F  Use 


eature 

RTU 

CTU 

Mean 

Reset 

4.3 

4.2 

4.3 

Total  System  Freeze 

4.5 

3.8 

4.2 

Recorded  Briefing 

1.2 

1.3 

1.2 

Demonstration 

2.2 

1.3 

1.9 

Record/Pl ayback 

2.3 

1.7 

2.1 

Environmental 

4.2 

3.4 

3.9 

Auto  Malfunction  Insertion 

3.3 

3.6 

3.4 

Flight  System  Freeze 

2.0 

3.2 

2.4 

Position  Freeze 

3.2 

2.8 

3.1 

Parameter  Freeze 

3.2 

2.9 

3.1 

Crash  Override 

4.2 

3.8 

4.1 

Hard  Copy 

3.3 

2.8 

3.1 

Programmed  Mission  Scenarios 

1.3 

2.3 

1.7 

Grand  Mean 


3.0 


2.9 


3.0 


Table  24.  E-3A  Flight  Simulator:  Mean  Amount  ot 
Training  Received  in  AIF  Use  (IPs) 


Feature  _  _ _ RTU _ CTU 


Reset 

4.4 

4.3 

Environmental 

5.2 

5.0 

Auto  Malfunction  Insertion 

4.0 

4.3 

Flight  System  Freeze 

5.2 

3.7 

Position  Freeze 

5.2 

3.3 

Parameter  Freeze 

4.0 

3.0 

Crash  Override 

5.4 

3.3 

Motion 

5.4 

6.0 

Hard  Copy 

1.0 

1.0 

Programmed  Mission  Scenarios 

2.2 

hi 

Mean 

4.4 


5.1 


4.1 


4.6 


4.5 


3.6 


4.6 


5.6 


1.0 


2.8 


Grand  Mean 


.2 


3.8 


4.1 


{ 


Table  25.  E-3A  Mission  Simulator:  Mean  Amount  of  Training 
Received  in  A1F  Use  (WDls) 


Feature 

RTU 

CTU 

Mean 

Total  System  Freeze 

1.5 

2.0 

1.7 

Recorded  Briefing 

1.3 

2.2 

1.5 

Demonstration 

3.0 

3.7 

3.2 

Record/Playback 

1.1 

1.3 

1.2 

Environmental 

1.8 

2.7 

2.0 

Crash  Override 

2.8 

2.7 

2.8 

Automated  Performance  Feedback 

2.4 

2.2 

2.3 

Hard  Copy 

3.3 

4.0 

3.5 

Programmed  Mission  Scenarios 

2.9 

4.0 

3.2 

Grand  Mean 

2.2 

2.7 

2.4 

Table  26.  Pooled  Data:  Mean  Amount  of  Training  Received  in  AIF  Use 


The  training  value  ratings  are  summarized  for  each  ATD-site  in  Tables  27 
to  31  and  for  the  pooled  data  in  Table  32.  The  individual  ratings  ranged  from 
1  (none)  to  7  (greatest).  Most  of  the  means  range  from  3.5  to  5.5  (i.e., 
moderate  to  considerable).  Not  surprisingly,  the  frequently  used  features 
were  also  those  rated  relatively  high  in  training  value.  A  notable  exception 
was  programmed  mission  scenarios,  which  was  rated  high  in  training  value  by 
all  instructors  except  the  A-10  RTU  IPs,  but,  was  not  used  often.  Another 
exception  was  hard  copy,  which  was  only  occasionally  used,  but  was  considered 
by  most  instructors  to  have  at  least  moderate  training  value.  An  interesting 
contrast  occurred  in  the  case  of  motion.  F-15  IPs  considered  it  to  have  only 
moderate  training  value,  whereas  E-3A  IPs  rated  it  highest  of  all  features. 

Like  the  ease-of-use  data,  the  training  value  data  were  a  function  of  the 
frequency  with  which  each  feature  was  used.  Thus,  RTU-CTU  overall  comparisons 
were  made  with  the  nonparametric  Wilcoxen  test,  whereas  comparisons  for  each 
feature  were  made  with  t-tests.  Among  the  various  ATDs,  the  only  significant 
overall  comparison  occurred  in  the  F-15  data.  In  this  case,  RTU  IPs  gave 
significantly  higher  ratings  than  did  CTU  IPs,  T  =  3,  £  <  .05.  Significant 
individual  comparisons  included  reset,  t(16)  =  4.02,  £  <  .001,  flight  system 
freeze,  t(32)  =  3.06,  £  <  .01,  and  motion.  t(27)  =  2.34,  £  <  .05.  The  only 
other  significant  differences  occurred  in  the  A-10  and  E-3A  IP  data: 
Environmental  was  rated  higher  by  A-10  RTU  IPs,  t(35)  =  2.56,  £  <  .05, 
programmed  mission  scenarios  was  rated  higher  by  A-10  CTU  IPs  t(12)  =  -  2.25, 

£  <  .05,  and  position  freeze  was  rated  higher  by  E-3A  RTU  IPs,  t(6)  =  2.74, 

£  <  .05. 

The  analysis  of  the  pooled  data  revealed  a  significant  overall  difference 
between  the  RTU  and  CTU  ratings,  T  *  21,  £  <  .05.  Apparently,  there  was  a 
general  tendency  for  RTU  instructors  to  rate  AlFs  higher  in  training  value. 
This  tendency  was  strongest  in  the  case  of  environmental,  t(116)  =  2.72, 

£  <  .01,  and  flight  system  freeze.  t(84)  *  2.27,  £  <  .05. 

A1F  Potential  Training  Value 

A  2  (training)  by  5  (ATD)  by  17  (AIF)  repeated  measures  analysis  of 
variance  (for  unequal  N)  was  used  to  analyze  these  data.  The  individual 
ratings  ranged  from  1  (none)  to  7  (greatest).  Like  the  training  value  data, 
described  previously,  most  of  the  means  range  from  3.5  to  5.5  (i.e.,  moderate 
to  considerable).  The  analysis  revealed  several  significant  effects. 


There  was  a  slight  but  significant  overall  difference  between  the  RTU 
(x  =  4.5)  and  CTU  (x  =  4.3)  ratings,  F (1 , 117 )  =  4.67,  £  <  .05.  The  main 
effect  of  ATD  was  not  significant,  however,  F(4,117)  =  1.77,  £  >  .05.  This 
reflects  the  fact  that  the  overall  mean  rating  was  similar  at  each  ATD  site. 
The  interaction  of  training  and  ATD  was  also  not  significant,  F( 4 , 117 )  =  2.41, 
p  >  .05.  This  reflects  the  fact  that  the  difference  between  the  RTU  and  CTU 
ratings  was  fairly  consistent  across  ATDs. 


Table  27.  F-4  Simulator:  Mean  Rated  A1F  Training  Value 


Feature 

RTU 

CTU 

Mean 

Reset 

5.4 

5.4 

5.4 

Environmental 

4.9 

4.1 

4.6 

Auto  Malfunction  Insertion 

4.2 

2.5 

3.8 

Flight  System  Freeze 

5.1 

4.5 

4.9 

Position  Freeze 

4.1 

4.2 

4.1 

Parameter  Freeze 

4.4 

4.6 

4.5 

Crash  Override 

5.4 

4.8 

5.2 

+  Auto  Performance  Feedback 

3.1 

3.3 

3.2 

Hard  Copy 

4.0 

4.7 

4.2 

+  Auto  Adaptive  Training 

4.0 

4.3 

4.1 

Programmed  Mission  Scenarios 

4^4 

5.0 

4.7 

Unweighted  Grand  Mean 

4.5 

4.3 

4.4 

+  Note:  These  AIFs  are  only  available  on  the  F-4E  simulator. 


Table  28.  F -15  Simulator:  Mean  Rated  AIF  Training  Value 


Feature 

RTU 

CTU 

Mean 

**  Reset 

5.6 

3.8 

4.9 

Environmental 

4.2 

3.6 

3.9 

Auto  Malfunction  Insertion 

4.9 

4.0 

4.4 

**  Flight  System  Freeze 

6.4 

5.1 

5.9 

Crash  Override 

5.3 

4.6 

5.0 

*  Motion 

3.6 

2.3 

T.l 

Hard  Copy 

3.6 

3.7 

3.7 

Programmed  Mission  Scenarios 

4.9 

5.4 

5.2 

Programmed  Mission  Scenarios 
Unweighted  Grand  Mean 


4.8 


4.1 


4.5 


Table  29.  A-10  Simulator:  Mean 

Rated  A1F 

Training 

Value 

Feature 

RTU 

CTU 

Mean 

Reset 

5.2 

5.3 

5.3 

Total  System  Freeze 

5.2 

5.0 

5.1 

Recorded  Briefing 

5.3 

2.5 

4.3 

Demonstration 

3.8 

3.3 

3.7 

Record/Playback 

4.1 

3.4 

**  Environmental 

5.1 

4.8 

Auto  Malfunction  Insertion 

3.6 

4.5 

3.9 

Flight  System  Freeze 

5.0 

4.6 

4.8 

Position  Freeze 

4.6 

4.5 

4.6 

Parameter  Freeze 

4.4 

4.1 

4.3 

Crash  Override 

4.9 

4.9 

Hard  Copy 

3.8 

*  Programmed  Mission  Scenarios 

2.8 

4.2 

3.6 

Unweighted  Grand  Mean 

4.5 

4.1 

4.4 

*  p  <  .05. 
**  p  < .025. 
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Reset 

5.8 

5.0 

5.5 

Environmental 

5.4 

6.0 

5.6 

Auto  Malfunction  Insertion 

5.6 

4.7 

5.2 

Flight  System  Freeze 

6.2 

4.7 

5.6 

Position  Freeze 

6.0 

4.7 

5.5 

Parameter  Freeze 

4.5 

5.3 

4.9 

Crash  Override 

5.4 

4.7 

5.1 

Motion 

6.8 

6.3 

6.6 

Hard  Copy 

1.0 

1.0 

Programmed  Mission  Scenarios 

4.0 

4^7 

4J> 

Programmed  Mission  Scenarios 
Unweighted  Grand  Mean 


5.1 


5.1 


5.0 


Table  31.  E-3A  Mission  Simulator:  Mean  Rated  A1F  Training  Value  (WDls 


Feature 

Rtu 

CTU 

Mean 

Total  System  Freeze 

3.2 

3.3 

3.2 

Recorded  Briefing 

3.6 

3.3 

3.5 

Demonstration 

5.3 

5.0 

5.2 

Record/P  layback 

4.6 

2.3 

3.8 

Environmental 

3.3 

3.3 

3.3 

Crash  Override 

4.7 

4.3 

4.6 

Automated  Performance  Feedback 

4.1 

5.0 

4.3 

Hard  Copy 

3.7 

3.2 

3.6 

Programmed  Mission  Scenarios 

4.6 

5.0 

4.8 

Unweighted  Grand  Mean 

4.1 

3.9 

4.0 

*: 


w 


4 


-‘4 


Table  32.  Pooled  Data:  Mean 

Rated  A1F 

Training 

Value 

feature 

RTU 

CTU 

Mean 

Reset 

5.3 

4.8 

5.1 

Total  System  Freeze 

4.6 

4.8 

4.7 

Recorded  Briefing 

3.8 

2.9 

3.5 

Demonstration 

4.4 

3.9 

4.3 

Record/Playback 

4.4 

3.9 

4.2 

**  Environmental 

4.7 

4.0 

4.4 

Auto  Malfunction  Insertion 

4.2 

4.2 

4.2 

*  Flight  System  Freeze 

5.5 

4.8 

5.2 

Position  Freeze 

4.5 

4.4 

4.5 

Parameter  Freeze 

4.3 

4.5 

4.3 

Crash  Override 

5.1 

4.7 

5.0 

Motion 

4.3 

3.4 

3.9 

Auto  Performance  Feedback 

4.0 

3.8 

3.9 

Hard  Copy 

3.8 

3.6 

3.8 

Auto  Adaptive  Training 

3.7 

3.6 

3.7 

Programmed  Mission  Scenarios 

4.4 

5.0 

4.7 

Unweighted  Grand  Mean 

4.4 

4.2 

4.3 

*  p  <  .05. 

**  p  <  .01. 
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There  was  a  significant  main  effect  of  A1F,  F(16,1872)  =  8.01,  £  <  .001, 
and  a  significant  interaction  of  Training  and  AlF,  F(16,1872)  =  2.25, 

£  <  .01.  The  relevant  data  are  summarized  in  Table  33.  The  Dunn  test  was 
used  to  analyze  the  main  effect  by  comparing  the  mean  ratings  of  all  pairs  of 
features.  Four  groups  of  features  were  identified  as  a  result.  The  highest 
ratings  were  assigned  to  reset,  record/playback.  and  programmed  mission 
scenarios.  Significantly  lower  ratings  (£<  .05)  were  assigned  to  total 
system  freeze,  demonstration,  auto  malfunction  insertion,  position  freeze. 
parameter  freeze,  auto  performance  feedback,  hard  copy,  and  auto  adaptive 
training.  The  lowest  ratings  (£  <  .01  relative  to  those  in  the  highest 
group,  £  <  .05  relative  to  those  in  the  intermediate  group)  were  assigned  to 
recorded  briefing  and  motion.  The  remaining  features  (i.e.,  instructor  pilot 
tutor i al,  environmental,  flight  system  freeze,  and  crash  override)  were  rated 
between  those  in  the  highest  and  intermediate  groups.  These  features  differed 
only  from  those  in  the  lowest  group,  £  <  .01.  It  is  especially  interesting 
to  note  that  of  the  three  highest  rated  features,  only  reset  was  frequently 
used. 

The  Dunn  test  was  also  used  to  determine  the  locus  of  the  Training  x  AIF 
interaction.  Only  one  significant  comparison  was  obtained,  that  for  auto 
performance  feedback.  Apparently,  neither  the  main  effect  of  training  nor  its 
interaction  with  AIF  was  very  robust. 

The  strongest  effect  obtained  (omega  squared  =  .07)  was  the  interaction 
between  ATD  and  AIF,  F(64,1872)  =  4.63,  £  <  .001.  Notwithstanding  the 
absence  of  a  significant  main  effect  of  ATD,  this  interaction  suggests  that 
there  might  be  significant  differences  among  the  ATD  instructors'  ratings  of 
certain  features.  The  relevant  data  are  summarized  in  Table  34.  Significant 
differences,  determined  by  the  Dunn  test,  were  revealed  for  those  features 
marked  with  an  asterisks.  The  probability  value  applies  to  the  largest 
observed  difference.  For  all  features  except  reset,  the  mean  rating  for  the 
E-3A  WDIs  was  significantly  lower  than  the  highest  observed  mean  rating.  In 
the  case  of  reset,  the  mean  rating  for  the  F-4  IPs  was  significantly  lower 
than  that  for  the  A-10  IPs,  and  in  the  case  of  motion,  the  E-3A  IPs  rating  of 
6.9  was  significantly  higher  than  all  others. 

Finally,  there  was  a  significant  triple  interaction,  F(64,1872)  =  2.10, 

£  <  .001,  which  suggests  that  the  pattern  of  RTU-CTU  differences,  shown  in 
Table  33,  was  not  consistent  across  all  ATDs.  It  is  unlikely  that  this  effect 
is  very  important  since  its  magnitude  is  quite  small  (omega  squared  =  .02). 

The  relevant  data  are  summarized  in  Tables  Cl  to  C5  in  Appendix  C. 

Interrelations  Among  the  Variables 


Table  35  shows  the  intercorrelations  among  the  ratings  of  each  feature  on 
each  of  the  five  questions.  All  the  coefficients  are  positive  and  significant, 
£  <  .001.  Thus,  a  feature's  rating  on  any  question  can  be  predicted  with 
greater  than  chance  accuracy  given  its  rating  on  any  other  question.  However, 
these  predictions  will  not  be  equally  precise.  The  respective  proportions  of 
variance  accounted  for  (indicated  by  the  squared  coefficients)  range  from  5% 
to  42%.  Since  the  coefficients  are  positive,  it  can  be  generally  stated  that 


33.  Potential  Value  of  AIFs:  Mean 
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Ratings  by 

RTU  and 

CTU  Instructors 

Feature 

RTU 

CTU 

Mean 

Instructor  Pilot  Tutorial 

4.8 

4.7 

4.7 

Reset 

5.0 

5.3 

5.1 

Total  System  Freeze 

4.2 

4.7 

4.3 

Recorded  Briefing 

3.9 

3.2 

3.6 

Demonstration 

4.6 

4.1 

4.4 

Record/Playback 

5.1 

5.2 

5.1 

Environmental 

4.8 

4.2 

4.6 

Auto  Malfunction  Insertion 

4.4 

4.4 

4.4 

Flight  System  Freeze 

4.6 

4.7 

4.6 

Position  Freeze 

4.5 

4.4 

4.5 

Parameter  Freeze 

4.3 

4.3 

4.3 

Crash  Override 

4.8 

4.6 

4.8 

Motion 

3.5 

3.4 

3.5 

Auto  Performance  Feedback 

4.4 

3.6 

4.1 

Hard  Copy 

4.5 

3.8 

4.3 

Auto  Adaptive  Training 

4.4 

3.7 

4.1 

Programmed  Mission  Scenarios 

5.1 

5,2 

5.1 

Grand  Mean 


4.5 


4.3 


4.5 


Table  34.  Potential  Value  of  AlFs:  Mean  Ratings  by  ATD  instructors 


feature 

F-4  IPs 

F -15  IPs 

A-10  IPs 

E-3A  IPs 

E-3A  WDIs 

Instructor  Pilot  Tutorial 

4.8 

4.5 

5.0 

5.1 

4.5 

*  Reset 

5.5 

5.1 

5.4 

5.7 

4.0 

**  Total  System  Freeze 

3.1 

4.6 

5.2 

4.6 

3.7 

Recorded  Briefing 

3.4 

3.3 

3.6 

4.0 

4.6 

Demonstration 

4.2 

4.1 

4.5 

4.1 

5.3 

Record/Playback 

4.8 

5.5 

4.9 

4.4 

5.7 

**  Environmental 

5.0 

4.4 

5.3 

5.3 

2.7 

**  Auto  Malfunction  Insertion 

4.2 

4.7 

4.6 

5.6 

3.6 

**  Flight  System  Freeze 

5.1 

5.6 

4.3 

5.6 

2.7 

*  Position  Freeze 

4.9 

4.2 

4.7 

5.6 

3.7 

Parameter  Freeze 

5.1 

4.0 

4.4 

4.9 

3.6 

*  Crash  Override 

5.4 

5.0 

4.7 

5.0 

3.7 

**  Motion 

3.4 

3.3 

4.0 

6.9 

1.8 

Auto  Performance  Feedback 

4.4 

3.8 

3.8 

4.0 

5.0 

Hard  Copy 

4.6 

4.3 

4.1 

3.9 

4.2 

Auto  Adaptive  Training 

4.6 

3.9 

3.9 

3.4 

4.7 

Programmed  Mission  Scenarios 

5.0 

5.4 

4.5 

5.3 

5.8 

Grand  Mean 

4.6 

4.5 

4.5 

4.9 

4.1 

*  p  <  .05. 

**  p  <  .01. 
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Table  35.  Matrix  of  Intercorrelations  Among  Frequency  of  Use, 
Ease-of-Use,  Training  Received,  Training  Value,  and 
Potential  Training  Value. 


FREQUSE 

EASEUSE 

TRECD 

TVALUE 

PTVALUE 

FREQUSE 

1.00 

EASE USE 

.51 

1.00 

TRECD 

.43 

.40 

1.00 

TVALUE 

.55 

.38 

.43 

1.00 

PTVALUE 

.42 

.23 

.33 

.65 

1.00 

Note:  All  correlations  are  significant,  j>  <  .001. 


the  greater  training  value  a  feature  has,  the  more  frequently  it  is  used,  the 
easier  it  is  to  use,  the  more  training  was  received  in  its  use,  and  the 
greater  its  potential  training  value. 

Table  36  summarizes  the  results  of  a  multiple  linear  regression  analysis 
in  which  the  frequency  of  A1F  use  is  predicted  from  a  linear  combination  of 
the  remaining  variables.  The  table  indicates  that,  together,  the  predictor 
variables  account  for  over  42%  of  the  variance  in  the  frequency-of-use 
ratings,  the  most  important  predictors  being  ease  of  use  and  training  value. 

Table  37  summarizes  the  results  of  a  multiple  linear  regression  analysis 
in  which  A1F  training  value  is  predicted  from  a  linear  combination  of  the 
remaining  variables.  The  table  indicates  that,  together,  the  predictor 
variables  account  for  over  53%  of  the  variance  in  the  training  value  ratings, 
the  most  important  predictor  being  potential  training  value,  followed,  in 
order,  by  frequency  of  use,  training  received,  and  ease  of  use. 


DISCUSSION 


This  section  briefly  discusses  the  utility  and  utilization  ratings  of  each 
A1F.  The  features  are  grouped  by  function,  just  as  they  were  in  Table  1.  The 
discussion  incorporates  the  instructors'  comments  as  well,  since  in  many 
cases,  these  comments  help  explain  the  ratings. 

Briefing  Features 

These  features  are  designed  for  briefing  the  student  and/or  instructor 
prior  to  or  during  a  simulator  training  session.  The  purpose  is  to  establish 
a  learning  set  and  to  increase  learning  readiness. 

Recorded  Briefing.  The  recorded  briefing  feature  is  available  only  at  two 
ATD  sites  and  it  is  used  very  rarely.  On  the  A-10  ATD  recorded  briefing  is 
available  only  for  student  IPs  as  part  of  simulator  checkouts.  Audio  visual 
media  are  limited.  E-3A  WDIs  indicated  that  recorded  briefing  is  too  difficult 
to  use  and  is  only  available  at  the  students  discretion.  Recorded  briefing 
apparently  has  greater  utility  for  RTU  training,  although  its  potential 
training  value  is  relatively  low.  Several  instructors  feel  this  feature  is 
unnecessary  and  prefer  to  brief  the  students  themselves. 

Demonstration.  Like  recorded  briefing,  the  demonstration  feature  is  used 
infrequently  and  is  presumed  to  have  greater  utility  for  RTU  training. 

However,  it  is  rated  as  having  somewhat  greater  training  potential  than 
recorded  briefing.  (One  A-10  IP  stated,  "There's  nothing  better  than  a  good 
demo.")  In  comparison  to  other  AIFs,  demonstration  is  time  consuming  to 
implement  on  both  the  A-10  and  E-3A  mission  simulators,  which  may  explain  its 
infrequent  use. 


Table  36.  Multiple  Linear  Regression  of  Frequency  of  A1F  Use  on 
Ease-of-Use,  Training  Received,  Training  Value,  and 
Potential  Training  Value. 


DEPENDENT  VARIABLE:  Frequency  of  AIF  Use 
MULTIPLE  R:  .6545 

MULTIPLE  R-SQUARE:  .4283 

ANALYSIS  OF  VARIANCE: 


SUM  OF  SQUARES 

OF 

REGRESSION 

1431.6965 

4 

RESIDUAL 

1910.8638 

1004 

PREDICTOR 

VARIABLE 

COEFFICIENT 

STO.  ERROR 

INTERCEPT 

-1.03064 

EASEUSE 

.38496 

.03308 

TRECD 

.17546 

.03534 

TVALUE 

.35176 

.04082 

PTVALUE 

.16036 

.04197 

STD.  ERROR  OF  EST.:  1.3796 


MEAN  SQUARE 

F  RATIO 

P 

357.9241 

188.059 

.0000 

1.9033 


STD.  REG 
COEFFICIENT 

t 

P 

.314 

11.639 

.0000 

.137 

4.965 

.0000 

.291 

8.617 

.0000 

.120 

3.821 

.0001 

DEPENDENT  VARIABLE:  AIF  Training  Value 


MULTIPLE  R:  .7312  STD.  ERROR  OF  EST.:  1.0292 


MULTIPLE  R-SQUARE:  .5346 


ANALYSIS  OF  VARIANCE: 


SUM  OF  SQUARES 

OF 

MEAN  SQUARE 

F  RATIO 

P 

REGRESSION 

1221.8385 

4 

305.4596 

288.362 

.0000 

RESIDUAL 

1063.5305 

1004 

1.0593 

PREDICTOR 

VARIABLE 

COEFFICIENT 

STD.  ERROR 

STD.  REG 
COEFFICIENT 

t 

P 

INTERCEPT 

.16897 

FREQUSE 

.19578 

.02272 

.237 

8.617 

.0000 

EASEUSE 

.10005 

.02610 

.099 

3.834 

.0001 

TRECO 

.14171 

.02631 

.134 

5.387 

.0000 

PTVALUE 

.53017 

.02673 

.479 

19.832 

.0000 

ALi 


*  *# 
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Instructor  pilot  tutorial.  Most  instructors  receive  some  formal  training 
in  AIF  use.  Typically,  this  training  is  included  as  part  of  the  instructor 
pilot  training  course.  However,  over  80%  of  the  comments  reveal  that  most  of 
the  training  in  AiF  use  tends  to  be  informal,  instructor  pilot  tutorial  is 
not  currently  available  at  any  of  the  surveyed  ATD-sites,  but  it  was 
consistently  rated  high  in  potential  training  value.  (One  instructor  felt  it 
was  unnecessary  and  stated,  "Hands-on  training  is  best.") 

Training  Management  Features 

These  include  various  features  designed  to  control  the  structure  and 
sequencing  of  tasks  within  a  training  session. 

Total  system  freeze.  Total  system  freeze  is  available  on  only  two  ATDs. 

If  given  a  choice,  most  instructors  apparently  prefer  to  use  flight  system 
freeze  to  suspend  simulated  flight.  A  total  system  freeze  on  the  E-3A  mission 
simulator  results  in  the  simultaneous  freezing  of  all  student  consoles, 
thereby  interrupting  the  entire  mission.  As  a  result,  E-3A  WDIs  especially 
dislike  this  feature  and  many  consider  it  unavailable  for  use.  In  contrast, 
on  the  A-10  ATD  total  system  freeze  is  easiest  of  all  freezes  to  use,  and  A-10 
IPs  use  the  feature  frequently. 

Reset.  Reset  is  among  the  most  frequently  used  and  highly  valued  features 
at  each  ATD  site.  It  is  typically  used  in  conjunction  with  flight  system 
freeze  and  permits  instructors  to  rapidly  re-initi alize  the  ATD  to  a  particular 
configuration  so  that  a  student  can  repeat  a  particular  maneuver  or  mission 
segment.  Not  surprisingly,  reset  is  used  especially  frequently  by  RTU 
instructors.  Reset  is  used  least  often  by  F-15  IPs,  who  commented  that  it 
requires  numerous  switch  actions. 

Crash  and/or  kill  override.  Used  mainly  for  convenience,  crash  override 
is  typically  left  "on"  at  all  times  so  that  an  instructor  can  avoid  losing 
data  or  having  to  reset  the  ATD  following  a  crash.  The  ratings  of  this 
feature  were  high  at  most  sites,  although  39%  of  the  E-3A  WDIs  considered 
crash  override  to  be  unavailable.  In  fact,  crash  override  is  frequently  used 
on  the  E-3A  mission  simulator.  However,  it  must  be  inserted  during  sim 
start-up,  and  WDIs  will  not  normally  insert  it  themselves. 

Automated  adaptive  training.  Automated  adaptive  training  is  available 
only  on  the  F-4E  simulator,  where  it  is  used  mainly  by  CTU  IPs.  It  was  rated 
as  difficult  to  use,  with  only  moderate  training  value.  Manual  override  of 
automated  adaptive  training  is  not  possible.  In  commenting  on  its  potential, 
several  instructors  doubted  its  training  effectiveness,  remarking  that  "it 
takes  the  IP  out  of  the  loop." 

Programmed  mission  scenarios.  Programmed  mission  scenarios  are  available 
at  all  sites  and  are  apparently  an  important  part  of  CTU  training.  Although 
they  were  rated  high  in  utility  (highest,  with  reset  and  record/playback,  in 
potential  training  value),  they  are  used  less  often  than  might  be  expected. 
Instructors'  comments  indicate  that  programmed  mission  scenarios  are  extremely 
"cumbersome  to  build"  and  relatively  difficult  to  use.  In  the  E-3A  flight 
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simulator,  programmed  scenarios  are  limited  to  programmed  malfunction  blocks, 
which  helps  explain  the  large  proportion  of  IPs  who  presumed  the  feature  was 
unavailable.  Many  CTU  instructors  would  like  to  have  a  greater  number  and 
variety  of  scenarios  available,  whereas  a  few  RTU  instructors  felt  the  feature 
leaves  the  IP  with  nothing  to  do. 

Variation  of  Task  Difficulty  Features 

This  group  of  features  permits  an  instructor  to  control  the  difficulty  of 
simulated  flight  through  variations  in  ATD  fidelity,  configuration,  or  task 
load  demands. 

Automated  malfunction  insertion.  Automated  malfunction  insertion  is 
available  on  four  of  the  five  ATOs  surveyed.  In  most  cases,  it  is  seldom 
used,  and  many  instructors  prefer  to  insert  malfunctions  manually.  Their 
comments  indicate  that  automated  insertion  is  sometimes  unreliable  and  can  be 
difficult  to  implement.  On  the  other  hand,  manual  insertion  is  easy  and 
immediate.  Automated  malfunction  Insertion  may  be  desirable  in  certain 
situations.  For  example,  the  typical  E-3A  flight  simulator  training  session 
emphasizes  emergency  procedural  training  and  E-3A  IPs  use  the  feature 
moderately  often. 

Environmental.  Some  capability  for  varying  environmental  condition  is 
present  at  each  ATD  site.  It  is  used  moderately  often  by  most  instructors, 
very  often  by  E-3A  IPs,  but  very  rarely  by  E-3A  WDIs.  The  E-3A  mission 
simulator,  which  can  only  simulate  winds  aloft,  has  the  most  limited 
environmental  capability  of  all  ATDs  surveyed.  This  helps  explain  why  a 
relatively  large  proportion  of  WDIs  reported  that  this  feature  is 
unavailable.  Other  instructors  expressed  the  need  for  increased  fidelity  and 
ease  of  use.  Environmental  was  rated  as  having  moderately  high  training 
value,  especially  for  RTU  training. 

Motion.  Motion  is  available  on  the  F -15  ATD,  where  it  is  rarely  used,  and 
on  the  E-3A  flight  simulator,  where  it  is  frequently  used.  This  difference 
apparently  reflects  a  difference  in  fidelity.  The  training  value  ratings 
suggest  that  motion  may  have  greater  utility  in  RTU  training,  but  its  overall 
potential  training  value  is  relatively  low.  (E-3A  WDIs  rated  motion  lowest  of 
all  AIFs.) 

Flight  system,  position,  and  parameter  freeze.  The  purpose  of  these 
features  is  to  permit  the  instructor  to  selectively  freeze  parts  of  the  ATD 
system  and  thereby  control  the  student's  task  load.  The  various  freezes  may 
be  viewed  on  a  continuum  bounded  by  total  system  freeze,  in  which  the  entire 
ATD  ceases  to  function,  at  one  end,  and  parameter  freeze,  in  which  a  single 
parameter  such  as  altitude  is  held  constant,  at  the  other.  At  most  sites,  the 
freezes,  particularly  flight  system  freeze,  are  used  frequently  and  are  rated 
high  In  training  value.  However,  they  are  used  not  so  much  to  reduce  task 
load  but,  rather,  to  suspend  temporarily  the  training  session  so  that  the 
instructor  can  offer  feedback  and/or  re- configure  the  ATD  for  another  task. 

Not  surprisingly,  the  utilization  and  utility  of  freeze  tend  to  be  lower  in 
CTU  training  because  of  the  extended  scenarios  that  characterize  CTU  training 
sessions. 
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Instructor  Monitor  and  Feedback  Features 


Although  none  of  these  features  were  included  in  the  survey,  several 
instructors  stressed  the  importance  of  instructor  feedback  in  the  form  of 
repeaters/annunciators  and  real-time  situational  displays. 

Student  Feedback  Features 


Student  feedback  is  one  of  the  most  important  determiners  of  effective 
learning.  These  features  are  designed  to  provide  this  feedback  in  various 
ways.  Ironically,  the  features  are  among  those  used  least  often. 

Record/Playback.  Record/Playback  is  available  on  the  A-10  ATD  and  the 
E-3A  mission  simulator.  It  is  rarely  used  at  both  sites  and,  compared  to 
other  AIFs,  is  relatively  difficult  to  use.  For  example,  on  the  A-10  ATD 
record/p layback  must  be  implemented  by,  first,  accessing  a  particular  CRT  page 
and,  second,  entering  playback  and  select  time  modes  by  means  of  a  light  pen. 
On  the  E-3A  mission  simulator,  record/playback  was  rated  most  difficult  of  all 
AIFs  to  use.  Many  WDIs  did  not  consider  it  to  be  fully  operational. 
Paradoxically,  record/playback  is  among  those  features  rated  highest  in 
potential  training  value. 

Automated  performance  feedback.  Automated  performance  feedback  is 
available  on  the  F-4E  ATD  and  the  E-3A  mission  simulator.  Although  45%  of  the 
instructors  indicated  that  automated  performance  feedback  was  unavailable,  the 
other  instructors  reported  using  it  moderately  often.  It  apparently  has 
greater  potential  utility  for  RTU  training.  A  few  instructors  commented  that 
this  is  a  useful  feature  only  if  manual  override  is  possible. 

Hard  copy.  While  a  potentially  useful  feature,  hard  copy  is  seldom  used 
at  most  sites.  It  has  not  been  operational  on  the  E-3A  flight  simulator  for 
several  years  and  is  rarely  operational  on  the  F-15  ATD.  Several  instructors 
at  other  sites  noted  problems  in  reliability  and  ease  of  implementation,  and 
one  instructor  commented  or  the  need  to  protect  classified  information. 

Graphic  situational  displays  are  apparently  preferable  to  numeric  summaries. 

A1F  Utilization 

The  results  of  this  survey  indicate  that  many  AIFs  are  not  used 
extensively.  Several  factors  appear  to  have  contributed  to  the  low  rate  of 
use.  First,  training  in  AIF  use  tended  to  be  minimal.  As  a  result,  some 
instructors  were  not  fully  familiar  with  the  AIF  capability  of  their 
respective  devices.  Second,  several  features  were  not  always  operational  due 
to  hardware  and/or  software  unreliability.  This  applied,  for  example,  to 
recorded  briefing  on  the  A-10  ATD  and  E-3A  mission  simulator,  auto  malfunction 
Insertion  on  the  F-4  ATDs  and  hard  copy  on  the  F-15  ATD  and  E-3A  flight 
simulator.  Many  instructors  considered  these  features  to  be  unavailable  for 
use.  Third,  some  of  the  more  complex  features,  such  as  demonstration, 
record/playback,  and  programmed  mission  scenarios,  took  relatively  longer  to 
implement.  Due  to  the  limited  amount  of  simulator  time  that  was  available  to 
train  students,  these  features  could  not  be  used  extensively.  The  fact  that 
these  features  were  nevertheless  rated  high  in  training  value  suggests  that 


infrequently  used  AIFs  can  be  effective  training  aids.  Finally,  there  were 
some  features,  such  as  environmental  and  total  system  freeze  on  the  E-3A 
mission  simulator,  that  were  either  too  limited  in  capability  or  deficient  in 
design  to  justify  frequent  use. 

The  multiple  regression  analysis  (Table  36)  suggests  that  ease  of  use  and 
training  value  are  the  most  important  factors  that  determine  AIF  use. 

However,  as  with  all  correlational  data,  cause-effect  relationships  have  not 
been  established.  Many  other  factors  are  doubtlessly  involved. 

RTU-CTU  Differences 

The  results  also  indicate  that  AIF  utility  and  utilization  differs  as  a 
function  of  training  unit.  This  difference  was  most  apparent  in  the  F-15 
data.  The  F-15  RTU  IPs  used  the  features  more  frequently,  found  them  easier 
to  use,  received  more  training  in  their  use,  and  rated  them  higher  in  training 
value.  Similar  tendencies  were  present  at  the  other  ATD  sites,  and  the 
analyses  of  the  pooled  data  suggest  that  AIFs  have  greater  utility  for  RTU 
training  in  general. 

The  relationship  between  AIF  utility/utilization  and  training  unit  is  not 
a  simple  one,  however.  Certain  features  were  rated  higher  by  RTU  instructors 
while  others  were  rated  higher  by  CTU  instructors.  In  most  cases  these 
differences  reflect  the  contrast  between  the  procedural-based  RTU  training 
session  and  the  scenario-based  CTU  training  session. 


CONCLUSIONS  AND  RECOMMENDATIONS 

A  broad  objective  of  this  investigation  was  to  provide  a  data  base  that 
would  help  in  defining  the  requirements  for  future  TAC  ATD  procurements  and 
help  in  developing  subsequent  ATD  training  programs.  The  data  suggest  that 
certain  AIFs  need  to  be  made  more  reliable  and  user-friendly  before  their 
training  effectiveness  can  be  ascertained.  Some  of  these  features  may 
eventually  prove  unnecessary.  It  is  clear,  however,  that  most  instructors 
have  not  yet  fully  explored  the  existing  instructional  capabilities  of  ATDs. 

It  is  recommended  that  formal  intensive  training  programs  be  established 
for  ATD  instructors.  These  programs  should  not  only  teach  instructors  how  to 
use  AIFs,  but  more  importantly,  how  to  use  them  effectively  -  that  is,  when 
and  why  to  use  them.  (Interestingly,  this  recommendation  is  implicit  in  the 
high  potential  training  value  ratings  of  instructor  pilot  tutorial.)  The 
principles  of  effective  AIF  use  still  need  to  be  specified,  however.  Such 
principles  will  not  be  derived  from  surveys,  but  rather,  from  empirical 
investigations. 
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ADVANCED  INSTRUCTIONAL  FEATURES  -  IP  SURVEY 


Name 

Rank  Sauadron 

Date 

FLYING  EXPERIENCE: 

Aircraft 

Total  Hours 

IP  Hours 

SIMULATOR  EXPERIENCE: 
Simulator 


Total  Hours 


IP  Hours 


BRIEFLY  DESCRIBE  A  "TYPICAL"  TRAINING  SESSION  ON  THIS  SIMULATOR: 


Please  familiarize  yourself  with  these  instructional  features  and  their 
definitions: 

Instructor  Pilot  Tutorial  -  provides  the  IP  with  self-paced  programmed 
instruction  in  the  capabilities  and  use  of  the  flight  simulator. 

Reset  -  permits  instructor  to  "return"  the  simulated  aircraft  to  a  stored 
set  of  conditions  and  parameters. 

Total  System  Freeze  -  permits  instructor  to  interrupt  and  suspend 
simulated  flight  by  freezing  all  system  parameters. 

Recorded  Briefing  -  permits  instructor  to  provide  student  with  information 
about  a  structured  training  session  through  audio/visual  media 
presentation. 

Demonstration  -  permits  instructor  to  demonstrate  aircraft  maneuvers  by 
prerecording  and  subsequently  playing  back  a  standardized  segment  of 
simulated  flight. 

Record/Playback  -  permits  instructor  to  record  and  subsequently  playback 
all  events  that  occurred  during  a  segment  of  simulated  flight. 

Environmental  -  permits  instructor  to  vary  environmental  conditions  such 
as  wind  direction  and  velocity,  turbulence,  temperature,  and  visibility. 

Automated  Malfunction  Insertion  -  permits  instructor  to  pre-program  a 
sequence  of  aircraft  component  malfunctions  and/or  emergency  conditions. 

Flight  System  Freeze  -  permits  instructor  to  simultaneously  freeze  flight 
control  and  propulsion  systems,  latitude,  longitude,  altitude,  and  heading. 

Position  Freeze  -  permits  instructor  to  simultaneously  freeze  latitude  and 
longitude. 

Parameter  Freeze  -  permits  Instructor  to  freeze  any  one  or  combination  of 
flight  parameters. 

Crash  and/or  Kill  Override  -  permits  instructor  to  allow  simulated  flight 
to  continue  without  interruption  following  a  "crash"  or  "kill." 

Motion  -  permits  instructor  to  vary  platform  motion  system  cues,  such  as 
roll,  pitch,  lateral,  and  vertical. 

Automated  Performance  Feedback  -  provides  student  with  visual  and/or 
auditory  signals  (including  verbal  messages)  that  identify  performance 
deficiencies. 
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Hard  Copy  -  provides  a  record  of  alphanumeric  and/or  graphic  performance 
data  from  the  automated  performance  measurement  system  for  debriefing 
purposes. 

Automated  Adaptive  Training  -  computer-controlled  variations  in  task 
difficulty,  complexity,  and  sequence  based  on  pilot's  performance. 

Programed  Mission  Scenarios  -  computer-controlled  standardized  training 
sessions  based  on  pre-programmed  event  sequences. 


1.  How  often  have  you  used  each  instructional  feature? 
(Check  the  appropriate  space.) 


/moderately/  /  very  / 

unavailable/never/rarely/occasionally/  often  /frequently/frequently/most  often 

Feature  / _ /  / _ /_ _ / _ / _ / _ / _ / 

0  1  2  3  4  5  6  7 


Comments: 


2.  How  easy  is  it  to  use  each  instructional  feature? 

(Check  the  appropriate  space.) 

never  used  or/  most  /  very  / 

unavailable/difficult/difficult/difficult/moderate/  easy  /very  easy/easiest/ 

Feature  / _ / _ / _ / _ / _ / _ / _ / _ / 

0  1  2  3  4  5  6  7 

Comments: 
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3.  How  much  training  did  you  receive  in  the  use  of  each  instructional  feature? 
(Check  the  appropriate  space.  Please  comment  as  to  whether  the  training  was 
formal  or  informal. ) 


unavailable/  none  /minimal/  some  /moderate/considerable/great/greatest/ 

Feature  / _ II  II _ / _ /  / _ / 

0  1  2  3  4  5  6  7 

Comments: 


4.  Rate  the  training  value  of  each  instructional  feature. 

(Check  the  appropriate  space.) 

never  used  or/ 

unavailable/  none  /minimal/  some  /moderate/considerable/great/greatest/ 

Feature  / _ /  / _ / _ i _ / _ /  / _ / 

0  1  2  3  4  5  6  7 

Comments: 


5.  Rate  the  potential  training  value  of  each  instructional  feature,  including  those 
you  are  not  familiar  with.  Assume  that  you  have  had  no  experience  using  any  of 
the  features  and  that  all  of  them  are  equally  easy  to  use.  Therefore,  base  your 
ratings  on  the  feature  definitions  alone.  (Check  the  appropriate  space.) 


none  /minimal/  some  /moderate/considerable/great/greatest/ 

Feature  / _ / _ / _ / _ /  / _ / 

1  2  3  4  5  6  7 

Comments: 


65 


Instructor  Pilot  Tutorial 
Reset 

Total  System  Freeze 
Recorded  Briefing 
Demonstrati  on 
Record/Playback 
Environmental 

Auto  Malfunction  Insertion 
Flight  System  Freeze 
Position  Freeze 
Parameter  Freeze 
Crash  Override 
Moti on 

Auto  Performance  Feedback 
Hard  Copy 

Auto  Adaptive  Training 
Programmed  Mission  Scenarios 


Table  C-l.  F-4  Simulator:  Potential  Training  Value  of  AIFs 


Feature 

RTU 

CTU 

Mean 

Instructor  Pilot  Tutorial 

4.6 

5.2 

4.8 

Reset 

5.5 

5.4 

5.5 

Total  System  Freeze 

2.6 

4.2 

3.1 

Recorded  Briefing 

3.3 

3.6 

3.4 

Demonstration 

4.1 

4.5 

4.2 

Record/P layback 

4.5 

5.5 

4.8 

Environmental 

5.0 

5.1 

5.0 

Auto  Malfunction  Insertion 

4.6 

3.4 

4.2 

Flight  System  Freeze 

5.1 

5.2 

5.1 

Position  Freeze 

4.7 

5.2 

4.9 

Parameter  Freeze 

5.1 

5.2 

5.1 

Crash  Override 

5.5 

5.2 

5.4 

Motion 

3.2 

3.8 

3.4 

Auto  Performance  Feedback 

4.4 

4.5 

4.4 

Hard  Copy 

4.7 

4.4 

4.6 

Auto  Adaptive  Training 

4.6 

4.8 

4.6 

Programmed  Mission  Scenarios 

4.9 

5.0 

5.0 

Grand  Mean 


4.5 


4.7 


4.6 


Table  C-2.  F-15  Simulator:  Potential  Training  Value  of  AIFs 


Feature 

RTU 

CTU 

Mean 

Instructor  Pilot  Tutorial 

4.9 

4.1 

4.5 

Reset 

5.4 

4.8 

5.1 

*  Total  System  Freeze 

4.0 

5.3 

4.6 

Recorded  Briefing 

3.9 

2.7 

3.3 

Demonstration 

4.5 

3.6 

4.1 

Record /Pl^back 

5.4 

5.5 

5.5 

Environmental 

4.8 

3.9 

4.4 

Auto  Malfunction  Insertion 

4.8 

4.5 

4.7 

*  Flight  System  Freeze 

6.2 

4.8 

5.6 

Position  Freeze 

4.4 

3.9 

4.2 

Parameter  Freeze 

4.4 

3.6 

4.0 

Crash  Override 

5.4 

4.7 

5.0 

**  Moti on 

4.2 

2.4 

3.3 

Auto  Performance  Feedback 

4.3 

3.3 

3.8 

Hard  Copy 

4.5 

4.2 

4.3 

Auto  Adaptive  Training 

4.2 

3.6 

3.9 

Programmed  Mission  Scenarios 

5.2 

5.6 

5.4 

Grand  Mean 

4.7 

4.1 

4.5 

*  p  <  .05. 

**  p  <  .01. 


70  ’ 


'v>:v.y,;vy- 


4  . 


v  -V\>y ;  j 


Table  C-3.  A-10  Simulator:  Potential  Training  Value  of  AIFs 


Feature 

RTU 

CTl) 

Mean 

Instructor  Pilot  Tutorial 

5.0 

4.8 

4.9 

Reset 

5.2 

5.8 

5.4 

Total  System  Freeze 

5.3 

4.9 

5.2 

Recorded  Briefing 

3.7 

3.3 

3.6 

Demonstrati  on 

4.5 

4.5 

4.5 

Record/Playback 

4.7 

5.3 

4.9 

Environmental 

5.6 

4.6 

5.3 

Auto  Malfunction  Insertion 

4.4 

4.9 

4.6 

Flight  System  Freeze 

3.9 

5.2 

4.3 

Position  Freeze 

4.5 

5.1 

4.7 

Parameter  Freeze 

4.2 

4.9 

4.4 

Crash  Override 

4.6 

4.8 

4.7 

*  Moti  on 

3.5 

5.1 

4.0 

Auto  Performance  Feedback 

3.8 

3.8 

3.8 

Hard  Copy 

4.4 

3.5 

4.1 

Auto  Adaptive  Training 

4.0 

3.7 

3.9 

Programmed  Mission  Scenarios 

4.3 

5.0 

4.5 

Grand  Mean 

4.4 

4.7 

4.5 

Table  C-4.  E-3A  Flight  Simulator:  Potential  Training  Value  of  AIFs  (IPs) 


eature  RTU  CTU  Mean 


p  <  .05. 

p  <  .01. 
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Table  C-5.  E-3A  Mission  Simulator:  Potential 

Training 

Value  of  AIFs  (WDIs) 

■* 

Feature 

RTU 

TIT 

Mean 

•  \ 

Instructor  Pilot  Tutorial 

4.8 

4.5 

I 

/> 

Reset 

3.6 

5.4 

4.0 

Total  System  Freeze 

3.6 

3.8 

3.6 

;-.i 

*.  4 
'  ,  '  * 

Recorded  Briefing 

4.6 

4.4 

4.6 

'  *4 

| 

Demonstration 

5.1 
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